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Background / Context: 

Description of prior research and its intellectual context. 

Research on the effectiveness of educational interventions usually is based on samples of 
students who remain in the same school over time. In contrast, most students transfer schools at 
least once during their K-12 school career, not including normative transfers such as those from 
elementary to middle school (Rumberger, 2002). Even when looking at just the two years prior 
to the 1998 NAEP, one-third of fourth graders, 19 percent of eighth graders, and 10 percent of 
twelfth graders had changed schools at least once (Rumberger, 2002). Mobility is higher among 
low-income and minority populations (Rumberger, 2002). While many studies have investigated 
the relationship of student mobility with achievement (Alexander, Entwisle, & Dauber, 1996; 
Reynolds, Chen, & Herbers, 2009; Rumberger & Larson, 1998; Tucker, Marx, & Long, 1998), 
the degree to which this relationship might vary among schools has not been fully investigated; 
in other words, are some schools more effective with mobile students than others? 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The purpose of the present study was to investigate the effect of student mobility on the 
academic achievement of a representative sample of public school students in a mid-Atlantic 
state, demonstrating the use of CCMM modeling, and to examine whether there is significant 
variation among schools in terms of the impact of mobility on achievement. 

Setting: 

Description of the research location. 

(May not be applicable for Methods submissions) 

Data for the current study were obtained from a prior study of student mobility in a mid-Atlantic 
state that took place in 2001-2003. (A full description is available in Rogers, 2004.) 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

(May not be applicable for Methods submissions) 

The sample for the present study included 1,669 students who were enrolled in one of the study 
schools (see below for further detail). The present study focuses on the students in their sixth 
year of schooling. Characteristics of these students are displayed in Table 1. Nearly half (45.4%) 
had made non-promotional transfers between schools at least once during these 6 years; 8.5% 
had made 3 or more moves in just six years. Similar to national studies of mobility patterns, 
highly mobile students in the present sample were disproportionately poor and non-white. All but 
13 of the sample students were in 6 th grade (i.e., had never been retained in grade). Scores on 
year 6 CTBS as well as prior reading tests tend to decrease as mobility increases. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 
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(May not be applicable for Methods submissions) 


N/A 

Significance / Novelty of study: 

Description of what is missing in previous work and the contribution the study makes. 

Methodological studies have examined the statistical consequences of ignoring student mobility 
and have found that parameter estimates and causal inferences are likely to be significantly 
problematic (Chung, 2009; Grady & Beretvas, 2010; Luo & Kwok, 2012). Because students are 
nested within multiple schools rather than a single school, correctly analyzing student-level data 
requires the use of cross-classified, multiple membership (CCMM) non-hierarchical models. 
While the use of CCMM models has been demonstrated in a number of studies, only a few have 
investigated the effect of student mobility on academic achievement (e.g., Goldstein et al., 2007). 

Statistical, Measurement, or Econometric Model: 

Description of the proposed new methods or novel applications of existing methods. 

Details on the use of CCMM are provided in Browne (2012) and Goldstein (2003). This study 
applies these methods to investigate the impact of mobility on achievement and the variation in 
mobility gaps among schools. 

Usefulness / Applicability of Method: 

Demonstration of the usefulness of the proposed methods using hypothetical or real data. 

Using MLwiN version 2.27 (Rasbash et al., 2012), a series of multilevel models were fitted. 

First, a series of traditional multilevel models were fitted assigning students to the first school 
attended in Year 6, ignoring their potential membership to multiple schools during Year 6 as well 
as their membership in prior schools. 

Specifically, these traditional models predicted Year 6 CTBS reading score (Year6_Rdg) for 
student i in school j. Starting with an unconditional model: 


Year6_Rdgij = [C, + e,, 
Poj = Too + u 0 j 


u 0 j ~ N(0, c 2 u o) 


Then adding a series of student covariates: grade at Year 6, centered around 6 (students who had 
never been retained would be in 6 th grade); prior reading score (the student’s most recent reading 
score, standardized with a mean of 0 and standard deviation of 1); demographics (a set of 
dummy variables indicating free or reduced-price meals, special education services, limited 
English proficiency, minority, and female). Prior reading score was set to randomly vary among 
schools; the other Level 1 variables were fixed at Level 2. Finally, the variable indicating the 
number of non-promotional school transfers was added to the model, and set to random at Level 
2. The full traditional model thus was: 
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Year6_Rdgij = poj + Pij(Grade-6) + p 2j (Prior_Rdg) + Pi/FRPL) + p 4 j(SpecEd) + Psj(LEP) + 
P6j(Min) + p 7 j(Female) + Pgj(Moves) + ey 

Poj = Too + u 0 j 
Pij = Yio 

P2j = 720 + U 2 j 
p3j = 730 
p4j = 740 
p5j = 750 
p6j = 760 
p7j = 770 
Psj = 780 + Ugj 

School and student-level residuals were assumed to be normally distributed with means of 0 and 
variances o 2 uq and c 2 e , respectively. Next, a similar set of models was fitted using cross- 
classification and multiple membership. First, a naive 3 -level model was run, nesting students 
within first prior-year school within first current year school. This is done in order to provide 
starting values for the CCMM which is to be run using Monte Carlo estimation (Browne, 2012). 
Then the cross-classified and multiple membership structure was specified, so that students are 
nested within Year 6 schools (cross-classified factor 1, allowing multiple membership) cross- 
classified by prior schools (cross-classified factor 2, allowing multiple membership). Monte 
Carlo burn-in was set to 500 and chain length to 150000. MLwiN default (non-informative) 
priors were used. The full CCMM model was: 

Year6_Rdgij = poj + Pij(Grade-6) + p 2j (Prior_Rdg) + P 3 j(FRPL) + p 4 j(SpecEd) + Psj(LEP) + 
P6j(Min) + p 7 j(Female) + Pgj(Moves) + e,, 

n _ „ ,v ,„(3) „(3) , v (2) „(2) 

P0j 700 ' -“j,Year6schs(i)W ijU Oj ■“j,Priorschs(i)W ijU Oj 

Pij = 7io 

n i y (3) (3) 

P2j — 720 ' -“j,Year6schs(i)W ijU 2 j 

p3j = 730 
p4j = 740 
P5j = 750 
p6j = 760 
p7j =770 

Q I V 1 (3) (3) 

P8j - 780 + ,Y ear6schs(i)W ijU 8j 

Research Design: 

Description of the research design. 

(May not be applicable for Methods submissions) 

The study utilized a quasi-experimental research design. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

(May not be applicable for Methods submissions) 
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As part of a larger study of student mobility in the state that took place in 2001-2003, data were 
collected from the cumulative hard-copy records of 7,803 students in 1 17 elementary, 1 10 
middle and 88 high schools from all 24 local education agencies (LEAs) in the state. These 315 
schools were randomly selected using proportional stratified sampling based on LEA and grade- 
level (elementary, middle, and high) enrollment. Special schools (e.g., alternative or special 
education) were excluded from the sample frame. From each school, one fifth-, eighth-, or 
twelfth-grade classroom was selected for student record review. Complete school histories were 
able to be obtained for 6,819 students (87.4%). Demographics of the student sample were not 
statistically significantly different from overall student demographics in the state. 

For the present study, students having complete school histories, a CTBS reading score at Year 6 
of schooling, and a prior reading score were selected (N=1669). The prior reading test score was 
based on the most recent score from among three possible assessments: CTBS, a state 
accountability test, or a state functional reading test. These scores were standardized to have a 
mean of 0 and a standard deviation of 1 within groups by test and grade. 

Findings / Results: 

Description of the main findings with specific details. 

(May not be applicable for Methods submissions) 

Results of both the traditional and CCMM models are displayed in Table 2 (Appendix B). 
Comparison of the traditional and CCMM model results shows that estimates of fixed effect 
parameters are similar. Estimates of random effects, however, differ in that where the traditional 
model allocates Level 2 variance to a single school (student’s first Year 6 school), the CCMM 
model distributes this variance among the set of Year 6 schools (u (3) ) and the set of prior-year 
schools (u (2> ). Thus, the proportion of total variance that is attributed to differences among 
schools (intra-cluster coefficient, ICC) is estimated by the CCMM model to be slightly higher 
(.285) than the traditional HLM model (.274). 

The present study investigated the effect of student mobility on reading achievement, and 
whether this effect varied significantly among schools. Focusing on the CCMM results (Table 3), 
6 th grade white male students in the sample, with average prior reading scores, who are not 
receiving any special services, and who have never had any non-promotional school transfers, 
are predicted to have a CTBS reading scale score of about 663. Holding all other factors 
constant, a single school transfer is estimated to predict a decline in scores by about .6; while this 
estimate is not statistically significantly different from 0, the variance for this estimate shows 
significant spread among Year 6 schools, f2 (3) 8,8 = 1 1.64 (SE 3.83). Comparing the partial model 
to the full model, the addition of the mobility variable improves model fit significantly 
(p<.0001). Assuming normality, we would expect 95% of the schools to have mobility slopes 
within the range -.611 ± 1.96(1 1.640) 1 2 . Thus, mobility gaps range from -7.298 to 6.076. 

In addition, there is a negative covariance between intercepts and fT slopes, estimated as -37.932 
(SE 12.276), indicating that schools with higher average reading scores tend to have steeper 
slopes; in other words, student mobility has an ever stronger impact on achievement in high- 
performing schools. 
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Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

The present study was limited in its lack of school-level measures for achievement and mobility. 
Indeed, inclusion of these contextual measures, if they were available, might alter the study 
findings (Raudenbush & Bryk, 2002). This study examined complete school history data from a 
statewide sample of students in order to investigate the relationship between mobility and 
reading achievement in the sixth year of schooling. Cross-classified, multiple membership 
models were used to accurately account for students’ membership in multiple schools during 
Year 6 as well as prior years. The relationship between mobility and reading scores was found to 
be non-significant on average, but examination of the variance components revealed that the 
impact of student mobility on reading achievement varied significantly among schools. 
Furthermore, the covariance estimate suggests that mobility gaps are especially large in schools 
with higher overall levels of achievement. This suggests that further research is necessary that 
more closely examines the contextual effects of mobility. 
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Not included in page count. 
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Appendix B. Tables and Figures 

Not included in page count. 


Table 1. Mean sample characteristics, by mobility level 



0 (N=912) 

1 (N=436) 

2 (N=179) 

3+ (N=142) 

CTBS reading 

665.532 (38.841) 

659.638 (41.099) 

653.743 (37.880) 

648.838 (35.547) 

Prior reading 

.120 (.956) 

.013 (.953) 

-.200 (.960) 

-.266 

(.932) 

6 th grade 

.997 (.057) 

.995 (.068) 

.978 (.148) 

.972 

(.166) 

FRPL 

.178 (.382) 

.222 (.416) 

.369 (.484) 

.465 

(.501) 

Special education 

.067 (.250) 

.085 (.279) 

.061 (.241) 

.070 

(.257) 

LEP 

.002 (.047) 

0 

.006 (.075) 


0 

Minority 

.310 (.463) 

.431 (.496) 

.553 (.499) 

.599 

(.492) 

Female 

.524 (.500) 

.498 (.501) 

.514 (.501) 

.585 

(.495) 


Note. Standard deviations are in parentheses. 


Table 2. CTBS reading scale scores in year 6, full model, traditional HLM vs. CCMM 


Variable 

Traditional HLM 

CCMM 

Fixed effects 



Intercept 

663.683 (1.900) 

663.381 (1.963) 

Grade-6 

9.726 (4.827) 

10.509 (4.948) 

Prior reading 

21.804 (1.009) 

21.667 (1.039) 

FRPL 

-6.018(1.801) 

-5.930(1.805) 

Special education 

-8.639(2.817) 

-8.374 (2.828) 

Limited English proficient 

-47.134(18.603) 

-48.064(18.943) 

Minority 

-6.562 (1.756) 

-6.292(1.769) 

Female 

1.348 (1.322) 

1.344 (1.317) 

School transfers 

-0.483 (0.657) 

-0.611 (0.664) 

Random parameters 



Year 6 school(s) 



o2(3), intercept 

261.664 (45.328) 

278.572 (51.861) 

g 2(3), prior reading 

46.340 (15.906) 

51.752 (15.316) 

o2(3), school transfers 

11.683 (6.451) 

11.640 (3.828) 

Prior schools 



o2(2), intercept 

N/A 

11.776 (14.952) 

Students 



o2e 

635.544 (25.004) 

624.347 (25.656) 

Total variance 

955.231 

978.087 

ICC 

0.274 

0.285 

Deviance 

15813.24 

15477.98 


Note. Standard errors are in parentheses. 
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Table 3. CCMM sequential models 


Variable 

Unconditional 

Partial 

Full 

Fixed effects 




Intercept 

653.422 (1.9) 

663.007 (1.816) 

663.381 (1.963) 

Grade-6 


9.506 (4.846) 

10.509 (4.948) 

Prior reading 


21.79 (1.041) 

21.667 (1.039) 

FRPL 


-6.354(1.794) 

-5.930(1.805) 

Special education 


-8.838 (2.852) 

-8.374 (2.828) 

Limited English proficient 


-47.08 (18.921) 

-48.064(18.943) 

Minority 


-6.381 (1.781) 

-6.292 (1.769) 

Female 


1.229 (1.33) 

1.344 (1.317) 

School transfers 



-0.611 (0.664) 

Random parameters 




Year 6 schools 




f2 <3> o,o, intercept 

497.725 (76.548) 

206.832 (38.569) 

278.572 (51.861) 

Q <3, 2 , 2 , prior reading 


50.24 (16.984) 

51.752 (15.316) 

f2 <3) 8,8, school transfers 



11.640 (3.828) 

f2 <3> o,8, intercepts/school transfers 



-37.932(12.276) 

Prior schools 




G <2> o,o, intercept 

90.436 (37.516) 

22.567 (18.263) 

11.776 (14.952) 

Students 




G e 0,0 

981.753 (39.603) 

635.883 (26.199) 

624.347 (25.656) 

Total variance 

1569.914 

915.522 

978.087 

ICC 

0.3170397 

0.225917 

0.285 

Deviance 

16233.061 

15508.272 

15477.98 


Note. Standard errors are in parentheses. 
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